Tail-pad report

1. What problem does tail-pad address?

The 1-bit estimator is built around an AVX-512 512-dimensional path.
When the input dimension is not a multiple of 512, the remaining non-full block has to be handled separately. In the original path, this tail part is processed in a more conventional way, which leads to lower SIMD utilization and extra overhead on the final block. As a result, the estimator becomes noticeably slower when the tail is large.

2. Current tail-pad method

The current tail-pad method directly pads the remaining dimensions to a full 512-dimensional block, so the tail can stay on the same AVX-512 path as the main body.

In other words:

• full 512-dim blocks use the regular AVX-512 estimator path,
• the final partial block is padded to 512 dims,
• the estimator can then continue using 512-bit vector loads, bitwise operations, popcount, and reduction without falling back to a less efficient tail path.

This keeps the implementation simple and improves the utilization of the AVX-512 path on non-512-aligned dimensions.

3. Single-function results

The following table reports median per-call latency for the single-function benchmark (warmup_ip_x0_q_512) on the OpenAI-1536 dimension sweep.

Overall, the gain is small when the dimension is already 512-aligned, but becomes significant when the remaining tail is large. The largest improvements appear when remaining_dim is close to 448, where the tail overhead is most visible in the original path.